Two objectives in the design of decision support systems (DSS) are to improve decision-making performance and to use DSS modeling forms that are natural, that is, to adopt modeling paradigms that are congruent with decision makers' conceptual models of decision tasks. By accomplishing the latter objective, a DSS should enjoy better conceptual ease of use and face validity. However, past research finds that DSS deemed natural for a task by decision makers, DSS designers, and researchers alike, often do not improve (or even hinder) performance; the inverse also occurs. Further, decision-making behavior seems quite sensitive to minor task differences. How reliably are decision mode/naturalness and performance related? This study utilizes the bootstrapping paradigm of psychological research to help answer this question. In assessing the naturalness and performance of differing model paradigms over time and across levels of task complexity, no single, systematic pattern emerges. But the results suggest that naturalness and performance are differentially sensitive to task contingencies. For example, while relative performance is stable over time only in the low complexity condition, relative naturalness is stable over time only in the intermediate complexity condition. One implication of the results is that conceptual ease of use may be an unreliable predictor of a DSS's effect on performance. DSS mechanisms may help decision makers better analyze model naturalness and performance.
There are three important considerations in DSS development. (1) Decision making involves both primary and secondary processes, where secondary processes concern selecting appropriate decision making tools, approaches, and information. (2) In making decisions, humans are subject to numerous cognitive limitations. (3) In order for end users to develop DSS, sophisticated, problem-oriented DSS generators must replace technically demanding DSS tools. These three considerations can be effectively addressed by including expert system components in DSS. An expert DSS for statistical analysis is proposed and used as an illustration. Decision making scenarios are used to indicate the potential of such a system. In particular, it appears that an expert DSS can provide support for both primary and secondary decision making and help ameliorate human cognitive limitations.